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Problem  Statement 


Develop  software  tools  to  support  analytical  data 
preparation  processes: 

-  Populate  authoritative  databases 

-  Prepare  data  for  analysis 

-  Post-process  model  output 


What  features  would  you  like  to  see  in  these  tools? 


-  Usability 

-  Reliability 

-  Flexibility 

-  Transparency 

-  Repeatability 


Introducing. . . 
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Process  Execution  Tool 


Choose  any 

import 

location 


Do 

Something 


MS  Access 
MS  Excel 
JOPES  B8 
SQL  Server 
XML 
Oracle 


Transform 
view,  modify,  add  and 
remove  data 


Choose  any 

export 

location 


using  tools 

Order  Of  Battle  -  Unit  -  Transform  -  Surrogation  -  GeoData Viewer  -  TPFDD 


Models 
MS  Access 
MS  Excel 
JOPES  B8 
SQL  Server 
XML 
Oracle 
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Process  Execution  Tool  (PET) 


•  PET  is  analogous  to  Microsoft  Project. 

-  A  graphical  display  of  the  tasks/steps  in  a  data  preparation 
process. 


•  PET  goes  a  step  further  than  MS  Project  by 
supporting  the  execution  of  process  steps. 

•  Process  steps  are  assigned  tools  (Step  Tools), 
which  complete  or  support  the  completion  of  the 
step. 
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Example  Process:  Populate  Authoritative  Database 


Receieve  Navy  Data  |lmport  Navy  Data  | 

Hie  ■ 


Perform  Data 
Validation  (Navy 
Data) 


Perform  Data 
Validation  Checks 
(Army  Data  file#!) 


|Receive  Army  Hie  #1 1 


Import  Army  Data 
Rle#1 


|Receive  Army  file  #2 1  |lmport  Army  file  #2  \ 


Perform  Data 
Validation  Checks 
(Army  file  #2) 


Merge  Army  Data 

Export  to 

Test  Developmental 

Port  Data  to 

— 1  •' 

files 

1 

Developmental 

Database 

Production  Database 

Database 
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Step  Tools 


Step  Tools  come  in  three  flavors: 

-  Generic  data  manipulation  tools 

•  To  import  data:  Data  Mapper 

•  To  filter,  merge,  modify,  and  export  data:  Transform  Tool 

•  To  create  translation  tables:  Surrogation  Tool 

-  Entity  specific  data  editors 

•  To  create  and  edit  units:  Unit  Tool 

•  To  create  and  edit  Orders  of  Battle:  OOBTool 

•  To  view  and  edit  entity  locations:  GeoData  Viewer 

•  To  view,  edit,  and  convert  TPFDs:  TPFD  Generator 

-  External  applications 

•  COTS  /  GOTS  software 

•  Your  own  custom  built  applications 
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Process  Execution  Tool  (PET) 


PET  should  be  useful  for  projects  or  processes  with: 

•  Many  complexly  sequenced  steps  (JWARS  TFPD) 

•  Multiple  data  sources  (CFDB) 

•  Multiple  data  output  formats  (JICM) 

•  Multiple  analysts  (MCS) 

•  Extended  Duration  (Analytical  Baseline) 

•  Changing  requirements 

Just  about  everything  we  do. 
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PET  Functionality 


Three  Levels  of  Functionality:  Process  Administrator,  Analyst, 
Observer 

•  Administrator  Functionality: 

-  Create  Process  -  Adding  and  Sequencing  Steps 

-  Assign  Step  Tools 

-  Assign  Step  Validation  Rules 

-  Designate  Automatic  Steps 

-  Establish  Connections  to  Source  Data 

-  Define  Transformations 

-  Assign  Analysts  to  Steps 

-  Monitor  Execution  and  Inspect  Intermediate  Data  Products 

-  Add  /  Modify  /  Delete  Steps  During  Execution 

•  Analyst  Functionality: 

-  Inspect  Intermediate  Data  Products  and  Data  Validation  Results 

-  Work  on  Assigned  Steps 

-  Enter  Step  Execution  Comments 

•  Observer  Functionality:  Monitor  Process 
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Benefits  of  Using  PEI 


To  Populate  Authoritative  Databases: 

1 .  Provide  visibility  into  process  completion. 

2.  Automate  data  consistency  checks. 

3.  Improve  process  efficiency. 

4.  Document  the  process. 
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Benefits  of  Using  PEI 


To  Post-Process  Model  Output: 

1 .  Facilitates  correlation  of  output  data  elements  within  a 
run. 

2.  Facilitates  comparison  of  output  data  between  runs. 

3.  Provides  framework  for  automating  MOE  calculation 
and  chart  generation. 

4.  Document  the  Process 
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Benefits  of  Using  PEI 


To  Prepare  Data  for  Analysis: 

1 .  Facilitates  controlled  modifications  for  analysis  of 
alternatives. 

2.  Facilitates  extending  analysis  using  consistent 
common  data 

3.  Document  the  process. 
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Example  Data  Preparation  Process 


Tidal  Wave  Mitigation  (TWM) 

A  Case  Study 

using  the  Process  Execution  Tool 


The  Case  Study  and  the  sample  data  is  available 

on  the  provided  CD 
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The  Case  Study  ... 


•  Demonstrates  the  overall  use  of  PET  from  data 
import  to  data  export. 

•  Demonstrates  the  benefits  of  using  PET  for 
managing  the  development  of  analysis  datasets. 

•  Is  based  on  an  entirely  fictional  and  unclassified 
scenario  and  unclassified  data. 

•  Consists  of  a  few... 
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...simple  steps... 


Import  OOB 
Format 


Import  MM 
Equipment 


Use  OOB 
Tool  to 
Generate 
Force 

Structure  for 
MM01 


Select 

TWM01  Force 
Structure 


Finalize 
Output  Fields 
forTWMOl 
Force 
Structure 


Export  MM 

ForcesTo 

Access 


Force  structure  with 
rolled  up  units 


Roll  Up  Units 

Identify 

Finalize  Units 

Export  MM 

and 

Rolled  Up 

List  Output 

Units  to 

Equipment  to 

Units 

Fields  for 

Access 

Third  Level 

TWM01 

Select  Rolled 
Up 

Equipment 


Make 

TWMUnitEqui 
P  File 


Rolled  up  ui 
aggregated 


nits  with 
equipment 


Down  Select 

equipment 

forMM-01 


Select  Large 
Items  Only 
less 

Personnel 


Final  TWM-01 

Equipment 

Fields 


Equipment  down 
only  what  is  playe 
TMW-01  Seen 


import 

Surrogate 

Li st  for  MM 
Equipment 

select 

Surrogates 
for  Service 
Code  'A' 

Select 
Updated 
Surrogates 
for  Service 
Code  'A' 


[EquipSurrog 

lationOroup-A 


Surrogate 
Translator- 
Select  Fields 


Substitute 
Surrogates 
for  Large  Unit 
Equipment 


Keep 

Sum 

FinalizeMM 

Surrogates 

Surrogates 

Unit  Equip 

by  Unit 

by  Unit 

Surrogates 

Export  MM 

Unit 

Surrogate 

Equipment  to 

Access 

select  to 
d  in  the 
ario 


Surrogation  of  Equipment 
Into  Representative  Types 
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...plus  a  few  more... 


Import 00B 
Format 


UseOOB 
Tool  to 
Generate 
Force 

Structure  for 
TWM01 


Export  TWM 

ForcesTo 

Access 


Import  TWM 
Equipment 


Down  Select 

equipment 

forTWM-01 

Select  Large 
Items  Only 
less 

Final  TWM-01 

Equipment 

Fields 

Personnel 

Import 

Select 

Make  an  ID 

Surrogate 

Make  the 

Add  Backun 

Select  Un- 

Surrogate 

Surrogates 

for 

Equipment  to 

Surrogated 

Surrogated 

Surrogated 

List  for  TWM 

forService 

Surrogates 

Surrogate  ID 

Equipment 

Equipment 

Equipment 

Equipment 

Code  'A' 

List 

Roll  Up  Units 

Identity 

and 

Rolled  Up 

Equipment  to 

Units 

Third  Level 

Finalize  Units 

Export  TWM 

List  Output 

Units  to 

Fields  for 

Access 

TWM01 

Step  Group 

For  Equipment 


Surrogation, 


Select  Rolled 

Make 

up 

TWMEquip 

Equipment 

File 

Import 

Select 

Updated 

Updated 

Surrogate 

14 

Surrogates 

List 

for  Service 

Code  'A' 

Update 

Recombine 

Update 

Surrogated 

Updated 

Surrogate 

Equipment 

Equipment 

Name 

Surrogates 

irrogate 
anslator- 
ilect  Fields 


Substitute 
Surrogates 
for  Large 
Equipment 


Surrogates 
by  Unit 


Sum 

Surrogates 
by  Unit 


FinalizeTWM 
Unit  Equip 


Export  TWM 
Unit 

Surrogate 
Equipment  to 
Access 


Show  the 
People  the 
Short  List  of 
Surrogates 
for  A 
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Initial  Data 


Force  List,  Tables  of  Equipment,  Equipment  Characteristics 
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Microsoft 


Output  Data 


Playable  units  and  equipment 


[Ready 

liBstart  |  ^Qj^^[wl[i][i][£l© 


|(51lnbo...|  [s|Micr...  |  l^jD:\D...|  ^Java...|  E?j"CA5...  |  J[®  Micr... 


MORS  Symposium  CG  C  WG  17 


18 


Microsoft 


Case  Study  “Results” 


•  Build  the  process: 

-  1-2  days  to  build  34  step  process. 

•  Execute  the  process  -  build  the  base  case: 

-  Less  than  2  hours 

•  Generate  excursion  data  set: 

-  Duration  depends  on  the  extent  and  location  of  changes 

-  Assertion  -  can  be  done  faster  and  more  reliably  with  PET 
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Questions? 
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Updates  and  Feedback 


SourceShare:  https://sourceshare.pae.osd.mil 

collaborative  software  development  site  (registration  required) 

Send  Comments  to: 

-  Thomas.Dufresne@osd.mil 
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